Papers with automatic systems
Dealing with Controversy: An Emotion and Coping Strategy Corpus Based on Role Playing (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Psychological studies aim at explaining internal mechanisms of emotions, while computational studies simplify them into labels. |
| Approach: | They propose to treat emotions as strategies to cope with salient situations . they introduce a task of coping identification and a corpus constructed via role-playing . |
| Outcome: | The proposed method allows to investigate the link between emotions and behavior, which also emerges in language. |
Can vectors read minds better than experts? Comparing data augmentation strategies for the automated scoring of children’s mindreading ability (2021.acl-long)
Copied to clipboard
| Challenge: | In-domain experts are recruited to reannotate augmented samples and determine to what extent each strategy preserves the original rating. |
| Approach: | They implement 7 different data augmentation strategies for the task of automatic scoring of children’s ability to understand others’ thoughts, feelings, and desires. |
| Outcome: | The data augmentation strategies outperform task-agnostic augmentations and automatic augmentation systems perform worst on the MIND-CA corpus. |
Effective Crowdsourcing for a New Type of Summarization Task (N18-2)
Copied to clipboard
| Challenge: | Currently, summarization research focuses on summarizing the entire text, but in practice, readers are often interested in only one aspect of the document or conversation. |
| Approach: | They propose a new task where the goal is to summarize a particular aspect of a document. |
| Outcome: | The proposed task is based on a crowdsourced data collection workflow that allows users to collect high-quality summaries. |
Multimodal Emoji Prediction (N18-2)
Copied to clipboard
| Challenge: | Emojis are small images that are commonly included in social media text messages. |
| Approach: | They propose a multimodal approach that is able to predict emojis in Instagram posts by using both text and image. |
| Outcome: | The proposed model incorporates both text and image to improve accuracy . |
Entity Tracking Improves Cloze-style Reading Comprehension (D18-1)
Copied to clipboard
| Challenge: | Recent work on reading comprehension tasks has improved with simple approaches, but still trail human performance. |
| Approach: | They propose to add additional entity features and a multi-task tracking objective to improve model performance . they compare the model's predictions with those of more complicated models . |
| Outcome: | The proposed model outperforms the current state of the art on the LAMBADA dataset by 8 pts. |
An In-depth Analysis of Implicit and Subtle Hate Speech Messages (2023.eacl-main)
Copied to clipboard
| Challenge: | Explicit hate speech is more easily identifiable by recognizing hateful words, but subtle messages are harmful . subtle messages contain linguistically subtle and implicit forms of HS, such as circumlocution, metaphors and sarcasm . social media have faced pressure from civil rights groups demanding to monitor and limit online hate speech . |
| Approach: | They propose to use a fine-grained definition of implicit and subtle messages to detect HS . they then experiment with neural network architectures to detect subtle content . |
| Outcome: | The proposed models perform satisfactory on explicit messages, but fail to detect subtle content. |
Automatic Detection of Generated Text is Easiest when Humans are Fooled (2020.acl-main)
Copied to clipboard
| Challenge: | Recent advances in neural language modelling make it possible to rapidly generate vast amounts of human-sounding text. |
| Approach: | They compare decoding methods with popular sampling-based decoding strategies . they show that multi-sentence excerpts can fool expert human raters over 30% of the time . |
| Outcome: | The proposed methods improve with longer excerpt length, but multi-sentence excerpts fool human raters over 30% of the time. |
Evaluating Gender Bias in Speech Translation (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing evaluation techniques for gender biases are lacking in the field of machine translation. |
| Approach: | They propose to use a free evaluation set to evaluate gender bias in speech translation. |
| Outcome: | The proposed set is the speech version of WinoMT, an MT challenge set. |
Movie101: A New Movie Understanding Benchmark (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods to narrate movies with no actors are difficult to implement in real situations . a new metric is proposed to provide the best correlation with human evaluation . |
| Approach: | They propose a large-scale Chinese movie benchmark to help visually impaired enjoy movies . they propose metric called Movie Narration Score (MNScore) which achieves best correlation with human evaluation. |
| Outcome: | The proposed method outperforms baselines and the existing methods. |
MPST: A Corpus of Movie Plot Synopses with Tags (L18-1)
Copied to clipboard
| Challenge: | a corpus of movie plot synopses and tags can be used to build automatic tagging systems . a method to collect these tags allows us to learn to predict tags from plot synoopsis . |
| Approach: | They propose to collect a corpus of movie plot synopses and 70 tags to analyze their properties. |
| Outcome: | The proposed method can be used to predict movie tags from plot synopses. |
Prototype-Based Interpretability for Legal Citation Prediction (2023.findings-acl)
Copied to clipboard
| Challenge: | citation prediction is a key problem in high-stakes decision making areas such as law . experts often require interpretability for automatic systems to be utilized in practical settings . |
| Approach: | They propose to use legal citation prediction to solve a problem with legal experts' feedback . they propose to add a prototype architecture to add interpretability while adhering to legal parameters . |
| Outcome: | The proposed model performs well while adhering to decision parameters used by lawyers. |
SRL4E – Semantic Role Labeling for Emotions: A Unified Evaluation Framework (2022.acl-long)
Copied to clipboard
| Challenge: | Existing datasets for emotion detection are heterogeneous in size, domain, format, splits, emotion categories and role labels, hampering progress in this area. |
| Approach: | They propose a framework for annotating emotions manually using a common labeling scheme to unify several datasets tagged with emotions and semantic roles. |
| Outcome: | The proposed framework unifies datasets tagged with emotions and semantic roles by using a common labeling scheme. |
Inquisitive Question Generation for High Level Text Comprehension (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing data-driven questions generate questions that fill gaps in knowledge . a dataset of 19K questions is used to generate meaningful questions . |
| Approach: | They propose a dataset of 19K questions that are elicited while a person is reading a document. |
| Outcome: | The proposed model generates reasonable questions, but the task is challenging. |
The Connection between the Text and Images of News Articles: New Insights for Multimedia Analysis (2020.lrec-1)
Copied to clipboard
| Challenge: | a case study of text and images reveals the inadequacy of simplistic assumptions about their connection and interplay. |
| Approach: | They propose to use a case study to analyze 1000 flood-related news articles . they find that articles cluster into seven categories related to different topical aspects of flooding . |
| Outcome: | The results show that flood-related news articles do not consistently report on a single, currently unfolding flooding event. |
On the Correlation of Word Embedding Evaluation Metrics (2020.lrec-1)
Copied to clipboard
| Challenge: | Word embeddings are geometrical representations of word paradigmatics and syntagmatics. |
| Approach: | They propose to investigate evaluation metrics on various datasets to find correlations . they propose a fast solution to select the best word embeddings among many others . |
| Outcome: | The proposed method could be used to select the best word embeddings among many others. |
On “Human Parity” and “Super Human Performance” in Machine Translation Evaluation (2022.lrec-1)
Copied to clipboard
| Challenge: | In this paper, we reassess claims of human parity and super human performance in machine translation. |
| Approach: | They reassess claims of human parity and super human performance in machine translation . they argue that human translation involves much more than what is embedded in automatic systems . |
| Outcome: | The proposed results show that human translation involves much more than what is embedded in automatic systems. |
Hybrid Uncertainty Quantification for Selective Text Classification in Ambiguous Tasks (2023.acl-long)
Copied to clipboard
Artem Vazhentsev, Gleb Kuzmin, Akim Tsvigun, Alexander Panchenko, Maxim Panov, Mikhail Burtsev, Artem Shelmanov
| Challenge: | Existing methods for text classification tasks are inherently ambiguous and can cause errors. |
| Approach: | They propose a method that combines epistemic and aleatoric uncertainty to estimate toxicity detection errors. |
| Outcome: | The proposed method outperforms existing methods for toxicity detection and other ambiguous text classification tasks. |
Human and System Perspectives on the Expression of Irony: An Analysis of Likelihood Labels and Rationales (2024.lrec-main)
Copied to clipboard
| Challenge: | a new study examines the recognition of irony by humans and automatic systems . a fine-grained annotation scheme allows for improved modeling of ironity in automatic systems. |
| Approach: | They propose a fine-grained annotation scheme that allows for better recognition of irony by humans and automatic systems. |
| Outcome: | The proposed model improves on tweets annotated with high confidence and agreement . it also performs better on high-confidence and highagreement samples compared to automated systems . |
Detecting Propaganda Techniques in Code-Switched Social Media Text (2023.emnlp-main)
Copied to clipboard
| Challenge: | a new study aims to detect propaganda in multiple languages using code-switching . social media platforms have made it easier for anyone to spread information to a wide audience . |
| Approach: | They propose to detect propaganda techniques in code-switched texts using a corpus of 1,030 texts . they propose to model multilinguality directly rather than using translation . |
| Outcome: | The proposed method combines different languages within the same text, presenting a challenge for automatic systems. |